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1I_1 . (SBU)  SUMMARY.  As  in  most  countries,  the  results 
of  public  opinion  polls  in  Slovenia  are  widely 
discussed  among  media,  politicians,  pundits  and  the 
public.  However,  few  of  these  "consumers"  are  aware  of 
the  methodologies  that  are  used  or  the  ease  with  which 
polling  data  can  be  manipulated.  Although 
misrepresentation  of  polling  data  exists  in  virtually 
every  country,  the  close  relationships  between  polling 
centers,  media  outlets,  and  political  parties  in 
Slovenia  make  it  incumbent  on  the  consumer  of  opinion 
polls  to  study  their  methodologies  closely  and  take  all 
results  with  a grain  of  salt.  In  an  analysis  of 
polling  methodologies  used  by  different  survey  research 
centers  in  Slovenia,  we  have  determined  that 
professional  pollsters  are  highly  knowledgeable  and 
cognizant  of  potential  biases  inherent  in  their  polling 
methods.  However,  lack  of  sufficient  funding  in  a 
small  country  often  means  that  polls  are  done  on  the 
cheap  and  that  many  of  the  results  contain  considerable 
bias.  More  difficult  to  determine  is  whether  bias  is 
ignored  intentionally  and  whether  political  parties  can 
"commission  bias"  when  paying  for  a poll.  END  SUMMARY. 

112.  (U)  In  late  August,  Post's  Public  Diplomacy  Officer 

(who  has  a PhD  in  Political  Science  and  a Master's- 
level  certificate  in  Survey  Research  Analysis) 
conducted  a series  of  interviews  with  researchers  at 
three  major  polling  centers  in  Slovenia:  the  Research 

Center  for  Public  Opinion  and  Mass  Communication  at  the 
University  of  Ljubljana;  the  DELO  STIK  polling  agency 
affiliated  with  the  leading  Slovenian  daily  "Delo";  and 
the  NINA  MEDIA  polling  agency  affiliated  with  the  daily 
"Dnevnik."  The  following  observations  and  comments  are 
based  on  the  interviews  mentioned  above  and  an  analysis 
of  the  methodologies  used  by  these  polling  centers. 


Sample  Bias  vs.  Sampling  Error:  A crucial  distinction 


13.  (U)  One  of  the  main  problems  with  public  opinion 
polls  in  Slovenia  and  one  of  the  main  causes  of  faulty 
analysis  by  the  media  lies  with  the  fact  that  polling 
agencies  often  do  not  take  adequate  care  to  ensure  that 
various  biases  are  eliminated  from  their  polling 
methodologies.  Every  opinion  poll  contains  some  bias 
and  reducing  bias  to  a minimum  often  comes  at  a great 
cost.  Thus,  every  polling  agency  faces  a tradeoff 
between  bias  and  the  cost  of  utilizing  expensive 
procedures  to  minimize  it.  Since  most  polling  agencies 
are  businesses  looking  to  make  a profit,  cost-cutting 
is  prevalent;  nevertheless,  lower  costs  often  mean  less 
accuracy. 

14.  (U)  Another  major  problem  is  that  the  media,  who 
are  the  primary  consumers  and  interpreters  of  opinion 
polls,  often  do  not  adequately  understand  the 
distinction  between  sample  bias  and  sampling  error. 
Opinion  polls  that  have  considerable  bias  but  that  have 
minimal  error  are  often  presented  to  the  public  as  if 
they  were  quite  accurate.  Typically,  for  example,  a 
media  outlet  will  report  that  an  opinion  poll  has  a 
margin  of  error  of  plus  or  minus  2-5  percent.  What 
they  do  not  say,  however,  is  that  in  addition  to  this 
margin  of  error  (known  as  "sampling  error"),  these 
polls  also — in  fact  invariably — contain  sample  bias, 
which  can  often  skew  a given  poll  by  as  much  as  10,  15, 
or  even  25  percent.  A recent  conversation  with  the 
news  director  of  a major  Slovenian  television  station 
revealed  a complete  failure  to  appreciate  this  crucial 
distinction . 

15.  (U)  Sampling  error  is  a measure  of  the  likelihood 
that  a perfectly  random  sample  of  the  population  will 
have  the  same  characteristics  as  the  population  as  a 
whole.  Sampling  error  can  be  measured  using  a 
statistical  formula.  Sample  bias,  however,  cannot  be 


measured.  Sample  bias  occurs  when  a polling  agency 
consistently  fails  to  obtain  perfectly  random  samples 
of  the  population  (often  for  a multitude  of  different 
reasons).  For  example,  if  a polling  agency  were  to 
interview  people  only  within  a 50-mile  radius  of  their 
headquarters,  they  would  consistently  fail  to  obtain  a 
perfectly  random  sample  of  the  population.  However, 
extensive  polling  within  this  50-mile  radius  could  very 


well  reduce  the  margin  of  error  to  plus  or  minus  2 
percent,  or  even  less. 

1.6.  (U)  Sample  bias  is  almost  impossible  to  eliminate. 

Seemingly  innocuous  methodologies  like  phoning  random 
numbers  to  conduct  surveys  can  produce  remarkably 
biased  results.  For  example,  poorer  households  tend 
not  to  own  telephones  (or  subscribe  to  fewer  lines),  so 
phone  interviews  consistently  bias  the  sample  in  favor 
of  wealthier  respondents,  who  in  turn  often  have 
different  voting  preferences.  On  the  other  hand,  if 
telephone  polls  are  conducted  during  the  day,  they  may 
consistently  exclude  working  professionals  (since  they 
are  less  likely  to  answer  their  home  phone)  and  thus 
overstate  the  views  of  the  elderly,  students,  and  the 
unemployed.  While  telephone  penetration  rates  in 
Slovenia  are  estimated  at  94  percent,  and  while  most  of 
the  centers  poll  from  3:00-9:00  PM,  the  prevalence  of 
telephone  bias  still  exists  and  impacts  polling 
results.  This  is  true  especially  since  "busier" 
members  of  household  are  less  likely  to  answer  the 
phone  while  those  who  have  more  free  time  are  likelier 
to  agree  to  participate  in  a survey. 

17.  (SBU)  Polling  agency  directors  were  of  course 
cognizant  of  such  methodological  flaws,  but  noted  that 
telephone  polls  were  cheaper  than  fieldwork  and 
indicated  that  the  exorbitant  costs  of  fieldwork  often 
prevented  them  from  doing  more  methodologically 
rigorous  polling.  Polling  on  the  cheap,  they  argued, 
was  better  than  no  polling  at  all.  While  it  is  true 
that  some  media  in  the  U.S.  regularly  conduct 
methodologically  substandard  polls,  these  polls  are 
usually  discounted  by  polling  professionals,  who  rely 
on  the  far  more  accurate  analyses  provided  by  academic 
institutions,  major  media  outlets,  and  reputed  polling 
organizations  like  Gallup,  who  have  the  financial  means 
to  conduct  methodologically  rigorous  surveys.  In 
Slovenia,  the  small  number  of  polling  centers  and  the 
modest  funding  they  receive  means  that  few  polls  employ 
a rigorous  methodology. 


Weighted  Samples  and  Representative  Regions 


18 . (U)  One  standard  method  for  reducing  bias  involves 
the  "weighting"  of  survey  samples.  This  involves 
factoring  into  the  analysis  the  known  differences 
between  the  sample  and  the  population  as  a whole 
(usually  determined  from  a census).  For  example,  if 
you  randomly  telephone  ten  people  and  the  result  is 
that  3 women  and  7 men  answer  the  phone,  and  yet  you 
know  the  population  is  split  evenly  between  men  and 
women,  then  to  weight  the  sample  you  would  simply 
multiply  the  responses  of  the  women  by  7/10  and  the 
responses  of  the  men  by  3/10.  The  problem  with  this 
method,  however,  is  that  while  it  may  be  easy  to  weight 
responses  by  gender  or  area  of  residence,  there  are 
hundreds  of  other  variables  that  simply  cannot  be 
weighted  because  their  underlying  values  are  unknown 
(e.g.  religiosity,  previous  affiliation  with  the 
Communist  Party,  etc).  Most  Slovenian  research  centers 
weight  their  samples  by  gender,  education,  and  region 
of  residence.  However,  other  variables  like  affluence, 
church  attendance,  and  previous  support  for  the 
Communist  regime  have  all  been  shown  to  have  a 
significant — and  perhaps  even  greater — impact  on 
political  preferences  than  the  variables  currently  in 
use . 

19.  (U)  Another  method  for  minimizing  bias  is  to  pick  a 
representative  region  based  on  prior  election  results. 
This  is  sometimes  called  the  "Peoria  method."  If  a 
particular  region  or  municipality  within  a country  had 
election  results  that  perfectly  mirrored  the  national 
averages,  then  polling  agencies  will  sometimes  conduct 
methodologically  rigorous  polls  within  those  regions  in 
order  to  predict  future  election  results.  The  problem 
with  this  method,  however,  is  that  populations  shift 
and  what  may  be  a representative  region  during  one 
election  cycle  may  no  longer  be  representative  during  a 
subsequent  cycle.  This  is  especially  true  if  a new  set 
of  issues  is  being  raised  from  one  election  to  the 

next . 


Slovenia's  EP  elections:  What  Went  Wrong 


II. 10.  (SBU)  The  failure  of  most  Slovenian  polling 
agencies  to  adequately  predict  the  European 
Parliamentary  elections  is  mainly  due  to  various 
different  types  of  sample  bias.  While  it  is  impossible 
to  authoritatively  determine  why  such  bias  occurred, 
one  hypothesis  that  seems  likely  is  that  the  winner  of 
the  EP  elections — Nova  Slovenija  (NSi  or  New  Slovenia)- 
-has  a rather  "unrepresentative”  electorate.  To  cite 
but  one  example,  NSi  attracts  both  less  educated  voters 
and  highly  educated  voters.  So,  if  a polling  agency 
samples  a population  and  weights  the  sample  by 
education,  they  may  actually  be  missing  the  two 
extremes — the  uneducated  and  the  highly  educated — and 
hence  the  result  may  under-represent  the  percentage  of 
voters  who  would  vote  for  NSi. 

III.  (SBU)  Anecdotal  evidence  from  field  polls 
conducted  by  DELO  STIK  also  indicates  that  New 
Slovenia's  voters  tend  to  be  more  fearful  of  the 
government  (a  relic  of  the  Communist  system),  and  hence 
less  likely  to  answer  public  opinion  polls  out  of  fear 
that  the  government  may  be  trying  to  keep  tabs  on  them. 
If  such  suspicions  really  do  exist,  they  could 
potentially  result  in  a large  understatement  of  the 
number  of  NSi  voters  within  a given  region.  This  sort 
of  bias  is  also  more  likely  in  a telephone  poll  than  in 
a field  survey. 

1L12.  (SBU)  The  inability  of  Slovenian  polling  agencies 
to  adequately  predict  the  turnout  for  the  EP  elections 
also  reflects  a different  type  of  bias,  one  associated 
with  the  way  the  survey  questions  are  phrased  and 
perceived  by  respondents.  It  is  conventional  wisdom 
that  when  asking  respondents  if  they  plan  to  vote  in  an 
election,  10-15  percent  more  respondents  will  claim 
they  intend  to  vote  than  actually  do.  Such  "social 
desirability  bias"  (as  it  is  commonly  known)  stems  from 
their  desire  to  appear  as  conscientious  citizens.  A 
poll  conducted  by  the  Center  for  Public  Opinion  and 
Mass  Communication  indicated  that  even  after  the 
election  was  over,  12  percent  more  respondents 
indicated  that  they  had  voted  than  actually  did.  While 
polling  center  researchers  were  well  aware  of  this  bias 
and  generally  predicted  the  low  turnout  in  the  EP 
elections  rather  accurately,  the  media  seemed  more 
surprised  that  so  many  fewer  voters  would  show  up  than 
had  claimed  they  would  vote. 

113.  (SBU)  A final  reason  for  the  poor  polling  results 
in  the  run-up  to  the  EP  elections  also  has  to  do  with 
the  problem  of  low  voter  turnout.  Many  of  the  polling 
center  directors  we  spoke  with  claimed  that  NSi  voters 
were  underrepresented  in  the  pre-election  voting 
because  they  are  much  more  disciplined  than  the 
electorates  of  other  parties  (meaning  that  they  are 
more  likely  to  show  up  on  election  day — rain  or  shine)  . 
Since  this  is  a quality  that  is  difficult  if  not 
impossible  to  measure  prior  to  election  day  itself,  the 
pre-election  polls  generally  overrepresented  the  other 
electorates  and  underrepresented  NSi. 


The  Tail  Wagging  the  Dog? 


1.14.  (SBU)  The  bias  inherent  within  different  polling 
methodologies  and  its  effect  on  the  popularity  of 
political  parties  is  not  unknown  to  polling  agencies. 
Researchers  at  the  Center  for  Public  Opinion  and  Mass 
Communication  told  us  that  without  properly  weighting  a 
sample  for  education,  the  results  consistently  favor 
right-of-center  parties.  From  these  and  other 
comments,  we  believe  that  the  effects  of  different 
methodologies  are  well  known  to  those  conducting  survey 
data  analysis  in  Slovenia. 

115.  (SBU)  Gossip  and  conspiracy  theories  abound  in 
Slovene  political  circles.  Polling  results  are  not 
immune.  The  chatter  that  we  hear  from  our  contacts, 
particularly  among  opposition  members,  is  that  polling 
is  notoriously  unreliable.  The  result,  they  say,  is 
that  the  ruling  coalition  gets  a boost  from  those  who 
want  to  go  with  the  winner.  According  to  one  source, 
the  ruling  LDS  party  commissions  their  own  "real"  polls 


for  internal  use  that  are  cannily  accurate,  predicting 
the  actual  July  European  Parliament  election  results  of 
one  party  to  within  half  a percentage  point. 


116.  (SBU)  The  director  of  Nina  Media  also  admitted 
that  he  was  personally  friendly  with  many  members  of 
LDS  and  that  they  frequently  joked  with  him  about  the 
results  of  his  polls.  Although  he  categorically  denied 
that  pressure  was  ever  put  on  him  to  fudge  the  results, 
he  admitted  that  most  political  parties  did  maintain 
regular  contact  with  his  polling  agency  and  that  they 
sometimes  commissioned  their  own  polls.  He  also 
confirmed  that  most  parties  are  keen  to  benefit  from 
higher  polling  results.  As  he  said,  "no  one  wants  to 
look  unpopular." 


Comment 


1[17.  (SBU)  The  level  of  knowledge  and  expertise  among 
professional  pollsters  in  Slovenia  is  very  high. 

However,  public  opinion  polls  are  extremely  easy  to 
manipulate  by  expert  pollsters.  Polling  agencies  that 
compete  for  contracts  have  an  incentive  to  maintain  a 
reputation  for  accurate  polling,  but  that  does  not  mean 
that  other  considerations  do  not  come  into  play.  In 
Slovenia,  polling  agencies  are  generally  eager  to  cut 
costs  and  conduct  polls  on  the  cheap.  Since  the  more 
accurate  methodologies  are  usually  the  most  expensive — 
requiring  field  interviews  and  persistence  in  tracking 
down  randomly  chosen  respondents — it  is  natural  for 
agencies  to  sacrifice  accuracy  in  order  to  cut  costs. 
However,  it  is  also  true  that  if  one  wanted  to  bias  a 
sample  in  favor  of  a particular  political  constellation- 
-either  left-of-center  or  right-of-center — it  would  be 
very  easy  to  do  so.  In  fact,  intentional  bias  of  this 
sort  could  easily  be  attributed  simply  to  the  high  cost 
of  a more  rigorous  methodology. 

1[18.  (SBU)  While  election  results  generally  validate 
the  accuracy  of  a given  polling  agency,  it  is  not 
impossible  for  inflated  or  deflated  pre-election  polls 
to  encourage  or  dissuade  potential  voters  and  thus 
influence  the  result  itself.  Generally,  therefore,  one 
needs  to  take  all  polling  results,  and  claims  of 
manipulation,  with  a grain  of  salt  and  be  aware  of  the 
fact  that  it  is  always  easy  to  explain  inaccuracies 
after  an  election  is  over.  Parliamentary  elections  in 
Slovenia  will  be  held  October  3. 

HAAS 


NNNN 


